Computational imaging is a new imaging paradigm that is capable of providing unprecedented user-experience and information based on images and videos. For example, computational imaging can process images and/or videos to provide a depth map of a scene, provide a panoramic view of a scene, extract faces from images and/or videos, extract text, features, and metadata from images and/or videos, and even provide automated visual awareness capabilities based on object and scene recognition features.
While computational imaging can provide interesting capabilities, it has not been widely adopted. The slow adoption of computational imaging can be attributed to the fact that computational imaging comes with fundamental data processing challenges. Oftentimes, image resolution and video frame rates are high. Therefore, computational imaging generally requires hundreds of gigaflops of computational resources, which may be difficult to obtain using regular computer processors, especially where that performance has to be sustainable and backed up by high memory bandwidth at low power dissipation. Furthermore, computational imaging is generally sensitive to latency. Because users are unlikely to wait several minutes for a camera to recognize an object, computational imaging cameras are generally designed to process images and videos quickly, which further burdens the computational requirement of computational imaging.
Unfortunately, it is difficult to implement computational imaging techniques in customized hardware. As the field of computational imaging is in its relative infancy, implementation techniques are in constant flux. Therefore, it is difficult to customize computational imaging entirely in hardware as changes to implementation techniques would require redesigning the entire hardware. Accordingly, it is generally desirable to provide a flexible hardware architecture and a flexible hardware infrastructure.
At the same time, the demand for such video and image processing is coming to a large extent from portable electronic devices, for example tablet computers and mobile devices, where power consumption is a key consideration. As a result, there is a general need for a flexible computational imaging infrastructure that can operate even under a constrained power budget.